Journey verification for ridesharing via audible signals

ABSTRACT

A system has a communication system that is in communication with a computing system. The computing system runs computer-executable instructions that may examine a profile of a passenger. Furthermore, the system may determine based upon the profile that support is to be provided audibly to the passenger whenever support is provided to the passenger. The system may also detect divergences from a predetermined path or route, which is determined may be determined by comparing a position signal output against the predetermined path. In response to detecting divergences, the system may further output an audible support message having content based on the detected divergences. Moreover, the system may be configured to contact authorities and stop the autonomous vehicle.

BACKGROUND

An autonomous vehicle is a motorized vehicle that can operate without ahuman driver. Conventionally, since the autonomous vehicle lacks adriver, physical human-machine interfaces (HMIs) may be provided toassist a passenger in finding the autonomous vehicle, requestingsupporting or information during a ride, and disembarking the autonomousvehicle safely. Exemplary HMIs may include a mechanical push-button, atouch-sensitive display, or the like. While such HMIs are well-suitedfor a large portion of the population, these HMIs may be sub-optimal forthose with vision impairments for a variety of reasons. For example, aperson with a visual impairment may have difficulty trying to locate theautonomous vehicle and verifying that the particular vehicle is thecorrect and/or intended vehicle that the person is looking for. Asanother example, a visually impaired passenger may have difficulty orconcerns during the ride as to the route and direction of the vehicle,in which case such HMIs may lack the ability to provide the visuallyimpaired passenger with desired support or information. Furthermore,upon disembarking from the autonomous vehicle may also prove to bedifficult. Without the assistance of a driver to guide the passengersafely out of the autonomous vehicle, such HMIs may be sub-optimal forvisually impaired passengers.

SUMMARY

The following is a brief summary of subject matter that is described ingreater detail herein. This summary is not intended to be limiting as toscope of the claims.

Described herein are various technologies pertaining to enablingridesharing and usage of an autonomous vehicle by a visually impairedpassenger.

More specifically, the various technologies present a notification tothe passenger based on a profile of the passenger, wherein the profileof the passenger specifies that the notification is to be providedaudibly to the passenger (e.g., the passenger may have a visualimpairment). Even more specifically, the various technologies presentthe notification when a divergence of the autonomous vehicle from aprior determined path is detected. Content of the notification is basedon the divergence of the autonomous vehicle from the prior determinedpath. Because the content of the notification is based on the detectionof divergence occurring, the notification can inform the visuallyimpaired passenger of the reason for the divergence. For instance, wherethe autonomous vehicle comes to a hard stop (e.g., has an observeddeceleration that is above a threshold), the notification can include“the car stopped because an animal ran into the road.” In anotherinstance, where the autonomous vehicle makes a turn instead ofcontinuing straight, the notification can include “the car has made aturn because the road is closed.”

In one example, a mobile computing device operated by the passenger ofthe autonomous vehicle provides an interface for presentingnotifications. The mobile computing device may be configured to have aglobal positioning system (GPS) or other position signal sensor tomonitor a location or position of the mobile computing device andassumptively the user of the mobile computing device. The mobilecomputing device may then detect divergences from a predetermined pathand, responsive to detecting the divergence, notify the passenger of thedivergence through a speaker (audible notification) or through avibration or pattern of vibrations of the mobile computing device(haptic or tactile notification and/or feedback).

In another example, a server is in communication with the autonomousvehicle and the mobile computing device of the passenger. The autonomousvehicle may include a sensor system that outputs a sensor signal that isindicative of a position or location of the autonomous vehicle. Acomputing system in the server detects divergences from thepredetermined path based on the sensor signal output from the sensorsystem. The computing system is then configured to notify the passengerof the occurrence of the divergence. The computing system may also beconfigured to alert an observer of the server or alert authorities ofthe divergence if the cause of the divergence is determined to bemalicious.

The above summary presents a simplified summary in order to provide abasic understanding of some aspects of the systems and/or methodsdiscussed herein. This summary is not an extensive overview of thesystems and/or methods discussed herein. It is not intended to identifykey/critical elements or to delineate the scope of such systems and/ormethods. Its sole purpose is to present some concepts in a simplifiedform as a prelude to the more detailed description that is presentedlater.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an environment having a user and an autonomousvehicle.

FIG. 2A illustrates an exemplary method of audible authenticationbetween a user and an autonomous vehicle.

FIG. 2B illustrates another exemplary method of audible authenticationbetween a user and an autonomous vehicle.

FIG. 3 illustrates an autonomous vehicle during a journey.

FIG. 4 illustrates a user and an autonomous vehicle arriving at adestination.

FIG. 5 illustrates an autonomous vehicle in accordance with this subjectdisclosure.

FIG. 6 illustrates a mobile computing device in accordance with thissubject disclosure.

FIG. 7 illustrates a server in accordance with this subject disclosure.

FIG. 8 is a flow diagram that illustrates various processes that occurwhen a user requests a ridesharing service.

FIG. 9A is a flow diagram that illustrates a pickup process in FIG. 8.

FIG. 9B is a flow diagram that illustrates a second safe boardingprocess.

FIG. 10 is a timeline flow diagram that illustrates a successful safeboarding process.

FIG. 11 is a schematic block diagram that illustrates a pickup process.

FIG. 12 is a flow diagram that illustrates a journey verificationprocess in FIG. 8.

FIG. 13 is a schematic block diagram that illustrates an enrouteprocess.

FIG. 14 is a flow diagram that illustrates a safe disembarking processin FIG. 8.

FIG. 15 is a schematic block diagram that illustrates a disembarkingprocess.

DETAILED DESCRIPTION

Various technologies pertaining to an autonomous vehicle are nowdescribed with reference to the drawings, wherein like referencenumerals are used to refer to like elements throughout. In the followingdescription, for purposes of explanation, numerous specific details areset forth in order to provide a thorough understanding of one or moreaspects. It may be evident, however, that such aspect(s) may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order tofacilitate describing one or more aspects. Further, it is to beunderstood that functionality that is described as being carried out bycertain system components may be performed by multiple components.Similarly, for instance, a component may be configured to performfunctionality that is described as being carried out by multiplecomponents.

Moreover, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom the context, the phrase “X employs A or B” is intended to mean anyof the natural inclusive permutations. That is, the phrase “X employs Aor B” is satisfied by any of the following instances: X employs A; Xemploys B; or X employs both A and B. In addition, the articles “a” and“an” as used in this application and the appended claims shouldgenerally be construed to mean “one or more” unless specified otherwiseor clear from the context to be directed to a singular form.

Further, as used herein, the terms “component” and “system” are intendedto encompass computer-readable data storage that is configured withcomputer-executable instructions that cause certain functionality to beperformed when executed by a processor. The computer-executableinstructions may include a routine, a function, or the like. It is alsoto be understood that a component or system may be localized on a singledevice or distributed across several devices. Further, as used herein,the term “exemplary” is intended to mean serving as an illustration orexample of something and is not intended to indicate a preference.

Disclosed are various technologies that are particularly well-suited foruse when a passenger of an autonomous vehicle has a vision impairment.More specifically, a computer-readable profile of a passenger canindicate that the passenger prefers to receive information audibly. Whenan event is detected that may be of interest to the passenger, theautonomous vehicle can cause an audible message to be presented to thepassenger. Hence, the passenger (who may be visually impaired) need notattempt to request information by interacting with a physicalhuman-machine interface (HMI).

Accordingly, sensed events on a trip in an autonomous vehicle cantrigger audible support for a passenger who has a visual impairment.While the techniques set forth herein are described for use withpassengers having visual impairments, it is to be appreciated that thesetechniques can be utilized to assist passengers having other types ofdisabilities and/or even without disabilities; thus, as noted herein,personalization settings in a profile of a passenger can indicate apreference for audible presentation of information when the passenger isin the autonomous vehicle.

Generally, an autonomous vehicle includes a display screen located in aninterior of the autonomous vehicle that can be configured to receivetyped-out support requests and provide support information. However,where the passenger is visually-impaired it can be difficult for thepassenger to operate the display screen to request support, much lessread the support information while the autonomous vehicle is moving.Thus, it may be preferable for the passenger to receive audible support.The disclosed methods and systems can be integrated with an autonomousvehicle to provide contextual audible support messages.

FIGS. 1-4 collectively show a high level overview of different processesthat may occur when a user uses an autonomous vehicle.

With reference now to FIG. 1, a user or passenger 102 may request use ofan autonomous vehicle 104. The user or passenger 102 may have a visionimpairment. Thus, typical use of and navigation to/from the autonomousvehicle 104 may be more difficult for the user 102. For instance, theuser 102 may try to follow a path 106 to a pickup location 108 to boardthe autonomous vehicle 104. However, due to the vision impairment, theuser 102 may have difficulty navigating to the autonomous vehicle 104.Thus, audible guidance 110 may be provided to the user to follow thepath 106 to the pickup location 108.

As the user 102 approaches the autonomous vehicle 104, the user 102 willneed to identify and authenticate the autonomous vehicle 104 todetermine that the particular autonomous vehicle 104 is the intendedautonomous vehicle to board. Due to the vision impairment the user 102may have difficulty with identifying the particular autonomous vehicle104 by traditional authentication methods, such as examining the licenseplate or make, model and/or color of the vehicle. Thus, the user mayaudibly authenticate the autonomous vehicle 104.

FIGS. 2A-2B show two different embodiments for audibly authenticatingthe autonomous vehicle 104.

FIG. 2A demonstrates a server 202 sending a first audible authenticationcode or signal 204 a over a first signal 206 a to the autonomous vehicle104 and a second audible authentication code or signal 204 b over asecond signal 206 b to a mobile computing device 200 of the user 102.The mobile computing device 200 of the user 102 may output the secondaudible authentication code 204 b so that the user 102 is audibly awareof and/or may audibly receive the second audible authentication signal204 b. The autonomous vehicle 104 then outputs the first audibleauthentication code 204 a so that the user 102 can audibly receive thefirst audible authentication code 204 a. The user 102 may then outputthe second audible authentication code 204 b so that the autonomousvehicle 104 can audibly receive the second audible authentication code204 b. The autonomous vehicle 104 can determine whether the audibleauthentication codes 204 a, 204 b match to form a successfulauthentication and accordingly unlock the autonomous vehicle. Theautonomous vehicle 104 may also then notify the server 202 of thesuccessful authentication over a third signal 208 a. Alternatively, theserver 202 may receive and confirm that the audible authentication codes204 a, 204 b match and accordingly unlock the autonomous vehicle 104over the third signal 208 a. Similarly, the mobile computing device 200of the user 102 may be configured to output the second audibleauthentication code 204 b so that the autonomous vehicle 104 may receivethe second audible authentication code 204 b, thus relieving the user102 from receiving and outputting the second audible authentication code204 b.

FIG. 2B demonstrates the server 202 sending the first audibleauthentication code 204 a over the first signal 206 a to the autonomousvehicle 104 and the second audible authentication code 204 b over thesecond signal 206 b to the mobile computing device 200 of the user 102.The mobile computing device 200 of the user 102 may output the secondaudible authentication code 204 b so that the user 102 is audibly awareof and/or may audibly receive the second audible authentication signal204 b. The autonomous vehicle 104 then outputs the first audibleauthentication code 204 a so that the user 102 can audibly receive thefirst audible authentication code 204 a. The user 102 then verifies thefirst audible authentication code 204 a against the second audibleauthentication code 204 b and notifies the server 202 and/or theautonomous vehicle 104 over the third signal 208 b that the audibleauthentication codes 204 a, 204 b match. The server 202 may then notifythe autonomous vehicle 104, so that the autonomous vehicle 104 wouldthen allow entry thereto.

According to some embodiments, the authentication codes may becommunicated in other ways as well (e.g., non-audible means). Forexample, in some embodiments, the server 202 may securely sendauthentication codes to the autonomous vehicle 104 and mobile computingdevice 200 of the user 102. The mobile computing device 200 may be incommunication with the autonomous vehicle 104 through an interface, suchas Bluetooth. Once the mobile computing device 200 is authenticated,connected to and/or in communication with the autonomous vehicle 104,then the mobile computing device 200 and the autonomous vehicle 104communicate to ensure that the user 102 is near the autonomous vehicle104. When the user 102 is near the autonomous vehicle 104, the mobilecomputing device 200 may signal to the autonomous vehicle 104 to openthe doors. In other words, the authentication codes are communicatedamong the autonomous vehicle 104, the mobile computing device 200, andthe server 202 non-audibly. More specifically, the authentication codesare sent from the server 202 to the autonomous vehicle 104 and themobile computing device 200. Then, the autonomous vehicle 104 and themobile computing device 200 communicate directly, so that the autonomousvehicle 104 and/or the mobile computing device 200 may authenticate theauthentication codes non-audibly. The autonomous vehicle 104 and/or themobile computing device 200 may then notify the other and/or the server202, which may then send notification of the successful authenticationto one or both of the autonomous vehicle 104 and the mobile computingdevice 200. In some embodiments, the autonomous vehicle 104 and themobile computing device 200 may communicate directly, so that theautonomous vehicle 104 and/or the mobile computing device 200 mayreceive the other authentication code to send both authentication codesback to the server 202 for verification. The server 202 would thennotify the autonomous vehicle 104 and/or the mobile computing device 200of the authentication.

In some scenarios, an unintended third party may also enter theautonomous vehicle 104 while the door is open. This may occur when theunintended third party enters before the user 102, after the user 102,and/or enters through a different door than the user 102. Thus, in someembodiments, the autonomous vehicle 104 may also track and verify anumber of people boarding the autonomous vehicle 104 against an intendednumber of users 102.

In some embodiments, the autonomous vehicle 104 may also verify that thedoors to the autonomous vehicle 104 are properly closed. If the doorsare not properly closed, the autonomous vehicle 104 may warn and/ornotify the user 102. For example, a user may have a long skirt, whichgets stuck in the door when the door closes. In some embodiments, theautonomous vehicle 104 may detect this and alert the user 102.

In some embodiments, the autonomous vehicle 104 may detect that the user102 is accompanied by young children. Thus, the autonomous vehicle 104may alert the user 102 to lock the doors near where the young childrenare sitting, so that the young children don't accidentally open thedoors. Moreover, the autonomous vehicle 104 may also detect that theyoung children are wearing seatbelts, such that if the autonomousvehicle 104 fails to detect the seatbelts being fastened, the autonomousvehicle 104 may alert the user 102.

Similarly, in some embodiments, the user 102 may also give commandsregarding these and other issues, while in the autonomous vehicle 104 orthrough their mobile computing device 200, so that the autonomousvehicle 104 may initiate the above.

FIG. 3 demonstrates that throughout the journey, the autonomous vehicle104 will have an actual journey route or path 304 that may have adivergence 306 from a predetermined journey route or path 302. Thus,after the user 102 boards the autonomous vehicle 104 and begins ajourney to a destination, the user 102 may be curious of thepredetermined journey route or path 302 that the autonomous vehicle 104should follow.

In some cases, the user 102 may also be curious of the reason for thedivergence 306 from the predetermined journey route 302. Similarly, itis important to know whether the divergence 306 from the predeterminedjourney route 302 is the result of a malicious attack from a thirdparty. Thus, as will be discussed in further detail below, a supportmessage 308 may be output so that the user 102 may be aware of thedivergence 306 and the cause of the divergence 306. The support message308 may be output audibly, haptically, visibly, and/or a variety ofother methods. Similarly, the autonomous vehicle 104 may provide thesupport message 308 for some and/or all maneuvers that the autonomousvehicle 104 takes. For example, “we are making a left turn at Bushstreet, there is a doubled parked vehicle in front of us. I will waitfor a safe opportunity and pass.” Thus, the support message 308 mayadditionally be used to let the user 102 know that there is not adivergence 306.

FIG. 4 demonstrates the autonomous vehicle 104 arriving at a destination402. In the absence of the divergence 306, the destination 402 will bethe intended destination as the user 102 requested. As the user 102prepares to disembark, the autonomous vehicle 104 may implementprocedures to ensure that the user 102 disembarks safely. For users 102with vision impairments, disembarking safely may be difficult due to alesser ability to see potential hazards or dangers outside of theautonomous vehicle 104. For example, a pedestrian 404 may be walkingdown the street towards the autonomous vehicle 104. Furthermore, theuser 102 may be unaware of which side of the autonomous vehicle 104he/she should disembark on. For example, in a typically unusualsituation, the autonomous vehicle 104 may pull over on the left side ofa one-way street. Thus, the autonomous vehicle 104 may provide anaudible message 406 on from a side that the user 102 should disembarkon. In some embodiments, the audible message 406 may provide directionsrelative to the user 102. For example, the audible message 406 may state“Please exit to your right.” In some scenarios, there may be multipleusers 102 in the autonomous vehicle 104. Thus, the autonomous vehicle104 may identify each user 102 and localize the audible message 406,such that the audible message 406 is sent to the specific user 102. Insome embodiments, the autonomous vehicle 104 may instead output theaudible message 406 having more generic instructions so that each user102 is notified of which side to exit from.

Similarly, the autonomous vehicle 104 may output the audible message 406to an exterior of the autonomous vehicle 104. The audible message 406may indicate that a person (e.g., a person that has a disability) isdisembarking from the autonomous vehicle 104 to notify and protectpotential pedestrians 404. Although described as audible, the audiblemessage 406 may be communicated in a variety of other methods, such ashaptically, visibly, etc.

Although various embodiments disclosed here relate to helping personsthat have disabilities, this and/or other embodiments may be used toassist any user 102. In other words, many of these improvements arestill helpful to people irrespective of disabilities.

FIG. 5 is a schematic block diagram of the autonomous vehicle 104. Theautonomous vehicle 104 may have an autonomous vehicle computing system502, an autonomous vehicle transceiver 512, an autonomous vehiclelocation detection system 514, an autonomous vehicle audio system 516, amicrophone 518, and an autonomous vehicle sensor system 520.

The autonomous vehicle computing system 502 may have an autonomousvehicle processor 504 and autonomous vehicle memory 506, where theautonomous vehicle memory 506 stores computer-executable instructionsthat are executed by the autonomous vehicle processor 504. As anexample, the autonomous vehicle processor 504 may be or may include acentral processing unit (CPU), a graphics processing unit (GPU), aplurality of GPUs and/or CPUs, an application-specific integratedcircuit (ASIC), or the like.

Within the autonomous vehicle memory 506 may be a support system 508 andcontrol system 510. The support system 508 is configured to receiveand/or store preference data for the user 102 in a computer-readableprofile of the user. The preference data may include or show that: 1)the user is differently abled (e.g. visual impairment); 2) informationshould be delivered in a certain way (e.g. audibly); and 3) thefrequency of update information (i.e. how often the autonomous vehicle104 should tell the user 102 of turns, stoplights, etc.). For example,the user 102 may have a visual impairment, so the user 102 would definein his profile that he would like information to be delivered audibly atevery turn.

Furthermore, the support system 508 may be configured to continuallytrack the movement of the autonomous vehicle 104 and provide updates ofthe movements of the autonomous vehicle 104 to the user 102. Forexample, the autonomous vehicle 104 may audibly notify the visuallyimpaired user 102 that the autonomous vehicle 104 is making a turn alongthe predetermined journey route 302 or is making a turn, resulting inthe divergence 306 from the predetermined journey route 302.

The autonomous vehicle memory 506 also stores computer-executableinstructions that run the control system 510. The control system 510 isconfigured to control a propulsion system 522, a braking system 524, anda steering system 526 of the autonomous vehicle 104. Similarly, thecontrol system 510 may also control other systems in the autonomousvehicle, such as locks on doors of the autonomous vehicle 104.

The autonomous vehicle 104 further has an autonomous vehicle transceiver512 that is configured to communicate with other systems. For example,the autonomous vehicle transceiver 512 may send to and/or receivesignals from a mobile device of the user and/or with the server 202.

Onboard the autonomous vehicle 104 may also be an autonomous vehiclelocation detection system or position sensor system 514. This may be anysensor that provides detection of the location of the autonomous vehicle104, such as a global positioning system (GPS) or a combination of othersystems.

The autonomous vehicle 104 may further have an audio system 516 that mayoutput information and other audible signals, such as music. The audiosystem 516 may also be a spatial audio system that provides audiblesignals from a variety of different directions, such that the user 102can understand directional cues from the audio system 516. For example,the audio system 516 may audibly provide directions from a first side ofthe autonomous vehicle 104 to exit the autonomous vehicle 104 from thefirst side.

Similarly, the autonomous vehicle 104 may have a microphone 518 and/orautonomous vehicle sensor system 520 to receive audible and other typesof signals. For example, the microphone 518 may receive theaforementioned second audible authentication code 204 b from the user.The autonomous vehicle sensor system 520 may be composed of a variety ofdifferent observational sensors, such as light detection and ranging(LIDAR) sensors, cameras, suspension monitors etc. These various sensorsmay provide a wide variety of signals that indicate different types ofmovement of the autonomous vehicle 104 as well as information about theenvironment around the autonomous vehicle 104. For example, thesuspension monitors may detect a turn based on different loads on eachwheel. As another example, cameras and LIDAR are able to determinewhether obstacles may be near a door of the autonomous vehicle 104 orwhether other hazards are near or in the surroundings of the autonomousvehicle 104.

FIG. 6 illustrates a mobile computing device 200 that the user 102 mayhave. The mobile computing device 200 may take form in a wide variety ofdifferent embodiments, including, but not limited to smart phones, smartwatches, tablets, hearing aids, etc. The mobile computing device 200 mayhave a mobile device processor 602, mobile device memory 604, a mobiledevice transceiver 608, a mobile device location detection system 610, amobile device audio system 612, and a mobile device microphone 614.

The mobile device processor 602 may be or may include a centralprocessing unit (CPU), a graphics processing unit (GPU), a plurality ofGPUs and/or CPUs, an application-specific integrated circuit (ASIC), orthe like.

The mobile device memory 604 stores computer-executable instructionsthat are executed by the mobile device processor 602. A supportapplication 606 may be stored within the mobile device memory 604. Thesupport application 606 may be configured to interface with the user 102to receive and/or store the preference data for the user 102 in acomputer-readable profile of the user. The preference data may includeor show that: 1) the user is differently abled (e.g. visual impairment);2) information should be delivered in a certain way (e.g. audibly); and3) the type of information to be given (i.e. guidance to and from theautonomous vehicle 104). For example, the user 102 may have a visualimpairment, so the user 102 would define in his profile that he wouldlike guidance to and from the autonomous vehicle 104 to be deliveredaudibly.

The mobile computing device 200 may have a mobile device transceiver 608configured to communicate with other systems. For example, the mobiledevice transceiver 608 may send to and/or receive signals from theautonomous vehicle 104 and/or with the server 202.

The mobile computing device 200 may also have a mobile device locationdetection system 610. This may be any sensor that provides a locationsignal of the autonomous vehicle, such as a global positioning system(GPS) or a combination of other systems.

The mobile device audio system 612 of the mobile computing device 200 isadapted to output audible signals, such as audible notifications, music,audible guidance, etc. The mobile device audio system 612 may be aspeaker, speakers, a headset, and/or other similar component or devicesthat provide audible signals. The mobile device audio system 612 may bea component within the mobile computing device 200 or may also be anattachment or peripheral that connects to the mobile computing device200.

The mobile computing device 200 may also have a mobile device microphone614. The mobile device microphone 614 is adapted to receive audiblesignals, such as the first audible authentication code 204 a from theautonomous vehicle.

FIG. 7 illustrates the server 202 that may communicate with theautonomous vehicle 104 and the mobile computing device 200. The server202 may have a server processor 702, server memory 704, a servercommunication system 712, and a data store 714.

The server processor 702 may be or may include a central processing unit(CPU), a graphics processing unit (GPU), a plurality of GPUs and/orCPUs, an application-specific integrated circuit ASIC), or the like.

The server memory 704 stores computer-executable instructions that areexecuted by the server processor 702. Within the server memory 704 maybe a pairing system 706, a route planning system 708, and a verificationsystem 710.

The pairing system 706 is configured to connect the user 102 with theautonomous vehicle 104. The connection may include both the designationor “pairing” of the user 102 with the specific autonomous vehicle 104and/or the authentication process detailed further below between theuser 102 and the specific autonomous vehicle 104.

The route planning system 708 is configured to determine thepredetermined journey route 302 based on a location of the user 102 atthe time of the user's request for ridesharing service and a location ofthe destination 402 requested. The route planning system 708 may takeinto account a wide variety of factors to determine the predeterminedjourney route 302, such as traffic, weather conditions, etc. The routeplanning system 708 may also be configured to determine the path 106 toassist the user 102 in finding the autonomous vehicle 104, as shown inFIG. 1. Similarly, the route planning system 708 may also be configuredto determine a final path from a drop-off location to the destination402, as shown in FIG. 4.

The verification system 710 is configured to track a location of theuser 102 through the mobile device location detection system 610 of theuser's mobile computing device 200 and a location of the autonomousvehicle 104 through the autonomous vehicle location detection system514. As the location of the user 102 changes, the verification system710 tracks the changes in real-time to form and update the actualjourney route 304. In addition, the verification system 710 isconfigured to compare the location of the user 102 and the actualjourney route 304 against the predetermined journey route 302 forpotential divergences 306.

The server communication system 712 configured to communicate with theautonomous vehicle 104 and/or the mobile computing device 200. Theserver communication system 712 may also be configured to communicatewith additional autonomous vehicles 104 a and other mobile computingdevices 200.

The server 202 may also have a data store 714, in which data for varioussystems may be stored. For example, the data store 714 may storeprofiles of users 102 and preference data for each user 102.Furthermore, data from autonomous vehicle sensor systems 520 may be sentthe data store 714 to be stored.

FIG. 8 is a flowchart diagram that provides an overview of a process theuser 102 may encounter when requesting and using a ridesharing service.Subprocesses within the process of requesting and using a ridesharingservice may consist of a pick-up process 806, an en route process 808,and a disembarking process 810. The process ends at step 812 if the user102 does not need the ridesharing service or does not indicate the needand/or preferences for audible support.

Part 1: Pick-Up

As mentioned above, FIG. 8 is a flowchart diagram that provides anoverview of a process the user 102 may encounter when requesting andusing a ridesharing service. At step 802, in various embodiments, theuser 102 requests the autonomous vehicle 104. At some earlier point intime, the user 102 may have set up a user profile, having preferenceand/or disability data therein. Thus, step 804 shows that upon receivingthe request, the server 202 may access the user profile and determinewhether the user is differently abled, requires special actions, and/orprefers special actions, such as providing information audibly.

FIGS. 9A and 9B each illustrate an exemplary pick-up process 806 a, 806b, respectively, that may be used for the pick-up process 806.

FIG. 9A illustrates what occurs after the user 102 requests use of theautonomous vehicle 104. As shown in step 902 a, the server 202 and/orthe mobile computing device 200 notifies the autonomous vehicle 104 ofuser profile information, which may include user preferences, specialactions, special actions and/or the user's 102 disabilities. Next step904 a shows the server 202 sends the first audible authentication code204 a to the autonomous vehicle 104 over the first signal 206 a and thesecond audible authentication code 204 b to the user 102 over the secondsignal 206 b. Although shown as occurring at different points in time,steps 902 a and 904 a may occur in any combination of points in time.For example, the two steps 902 a, 904 a may occur concurrently, or step904 a may occur before step 902 a. Similarly, the server 202 may sendthe audible authentication codes 204 a, 204 b in any order to eitherparty. For example, the first audible authentication code 204 a may besent to the user 102 after the second audible authentication code 204 bis sent to the autonomous vehicle 104.

In step 906 a, when the autonomous vehicle 104 arrives at pickuplocation 108, the autonomous vehicle 104 notifies the server 202 and/orthe mobile computing device 200 of the user 102 of the arrival of theautonomous vehicle 104. In some instances, the autonomous vehicle 104may be unable to arrive at the exact location of the user 102, whichwould require the user 102 to travel to the location of the autonomousvehicle 104 for pick-up and/or boarding.

For users 102 with visual impairments, finding the location of theautonomous vehicle 104 may be difficult without assistance. Thus, instep 908 a, if the user 102 has noted in a user profile that the user102 is visually impaired and/or would prefer to receive audible guidanceto the pick-up location 108, the server 202 and/or the mobile computingdevice 200 would provide audible guidance 110 for the path 106 from theuser's current position to pick-up location 108, where the autonomousvehicle 104 is situated.

FIGS. 2A and 9A show in step 910 a, when the mobile device locationdetection system 610 outputs a position signal similar to and/or withina threshold distance from the pick-up location 108, the autonomousvehicle 104 audibly outputs the first audible authentication signal 204a. At step 912 a and 914 a, the user 102 then hears the first audibleauthentication signal 204 a and verifies the first audibleauthentication signal 204 a against the second audible authenticationsignal 204 b that the server 202 sent to the user's mobile computingdevice 200 in step 904 a. In some embodiments, the mobile computingdevice 200 may output the second audible signal 204 b again so that theuser 102 may correctly verify the audible authentication codes 204 a,204 b. The user 102 may then notify the server 202 whether the audibleauthentication codes 204 a, 204 b match. In some embodiments, the user102 may select an option on the mobile computing device 200 indicatingwhether the audible authentication codes 204 a, 204 b match. If the user102 verifies that the audible authentication codes 204 a, 204 b match,then the process continues forward to step 918 a.

Step 916 a shows that if the audible authentication codes 204 a, 204 bdo not match, the autonomous vehicle 104 a is not the autonomous vehicle104 that the user 102 is searching for. Thus, the user 102 does notboard the autonomous vehicle 104 a and is prompted to search nearby forthe intended autonomous vehicle 104. More specifically, the server 202receives notice that the audible authentication codes 204 a, 204 b donot match. Thus, the server 202 sends instructions to the autonomousvehicle 104 and/or the mobile computing device 200 to notify the userthat the autonomous vehicle 104 is not the intended autonomous vehicle104 and to search nearby for the intended autonomous vehicle 104. Insome embodiments, the mobile computing device 200 may communicatedirectly with the autonomous vehicle 104 that the autonomous vehicle 104is not the intended autonomous vehicle 104. In some embodiments, tonotify the user 102 that the autonomous vehicle 104 is not the intendedautonomous vehicle 104, the autonomous vehicle 104 may issue a signal,such as a honk, to indicate that the autonomous vehicle 104 isincorrect. In some embodiments, the intended autonomous vehicle 104 mayissue a signal, such as a honk, to provide the user 102 notice of wherethe autonomous vehicle 104 is located. Again, audible guidance 110 maybe provided to the user 102. In some embodiments, the mobile computingdevice 200 of the user 102 could notify the user 102 in some way that isconsistent with the user's understanding that there is a mismatch withthe autonomous vehicle 104. The mismatch may also be verified by theserver 202 and sent to the mobile computing device 200. For example, themobile computing device 200 may also send, in combination with whetherthe audible authentication codes 204 a, 204 b match, the audibleauthentication codes 204 a, 204 b received to the server 202. The server202 may check and/or verify whether the audible authentication codes 204a, 204 b match. In some embodiments, the mobile computing device 200 maybe a smart braille device, such that the smart braille device cancommunicate to the user 102 the match or mismatch with the autonomousvehicle 104.

Matching of the audible authentication codes 204 a, 204 b may have avariety of different implementations. For example, a sequence of randomor specific letters, words, and/or numbers may be used (e.g. a licenseplate of the autonomous vehicle 104, 1234, 1493, ABC, ASDF, etc.). Forexample, in some embodiments, the server 202 may send the user 102 thelicense plate number as one of the authentication codes, such that ifthe user gets the same license plate number as one of the authenticationcodes from the autonomous vehicle 104, then the autonomous vehicle 104and the user 102 are matched. As another example, a melody of a song mayalso be used. Similarly, a successful authentication need not be exactcopies of the audible authentication codes 204 a, 204 b (e.g., the firstaudible authentication code 204 a may be 1234, while the second audibleauthentication code 204 b may be 5678). As another example, a first partof a melody of a song may be used as the first audible authenticationcode 204 a and a second part of the melody of the song may be used asthe second audible authentication code 204 b. In other words, a“matching” may be a relational connection between the first audibleauthentication code 204 a and the second audible authentication code 204b instead of a mirror image or identical copy of the other. Thus,matching of the audible authentication codes 204 a, 204 b to form thesuccessful authentication may be implemented in a variety of differentways.

At step 918 a, when the user 102 verifies that the audibleauthentication codes 204 a, 204 b match, the user 102 is allowed toboard the autonomous vehicle 104. More specifically, in response toreceiving notice that the audible authentication codes 204 a, 204 bmatch, the server 202 sends instructions to the autonomous vehicle 104and/or the mobile computing device 200 to notify the user 102 that theautonomous vehicle 104 is the intended autonomous vehicle. In someembodiments, the mobile computing device 200 of the user 102 maycommunicate securely with the autonomous vehicle 104 either directlyand/or through the server 202. Furthermore, when the user 102 is allowedto board the autonomous vehicle 104, the autonomous vehicle 104 maynotify the user 102. In some embodiments, the autonomous vehicle 104 mayunlock the doors, play a welcome message, welcome the customer with apersonalized message verifying the name of the customer and/or thedestination 402 etc. In some embodiments, the autonomous vehicle 104 mayrecognize that the user 102 has come with additional passengers andwelcomes the additional passengers as well. Step 920 a shows the user102 boarding the autonomous vehicle 104.

FIG. 9B also illustrates what may occur after the user 102 requests useof the autonomous vehicle 104. More specifically, FIG. 9B shows asimilar pick-up process with a different method of authentication. Asshown in step 902 b, the server 202 and/or the mobile computing device200 notifies the autonomous vehicle 104 of user profile information,which may include user preferences, special actions and/or the user's102 disabilities. Next step 904 b shows the server 202 sends the firstaudible authentication code 204 a to the autonomous vehicle 104 over thefirst signal 206 a and the second audible authentication code 204 b tothe user 102 over the second signal 206 b. Although shown as occurringat different points in time, steps 902 b and 904 b may occur in anycombination of points in time. For example, the two steps 902 b, 904 bmay occur concurrently, or step 904 b may occur before step 902 b.Similarly, the server 202 may send the audible authentication codes 204a, 204 b in any order to either party. For example, the first audibleauthentication code 204 a may be sent to the user 102 after the secondaudible authentication code 204 b is sent to the autonomous vehicle 104.

In step 906 b, when the autonomous vehicle 104 arrives at pickuplocation 108, the autonomous vehicle 104 notifies the server 202 and/orthe mobile computing device 200 of the user 102 of the arrival of theautonomous vehicle 104. In some instances, the autonomous vehicle 104may be unable to arrive at the exact location of the user 102, whichwould require the user 102 to travel to the location of the autonomousvehicle 104 for pick-up and/or boarding.

For users 102 with visual impairments, finding the location of theautonomous vehicle 104 may be difficult without assistance. Thus, instep 908 b, if the user 102 has noted in a user profile that the user102 is visually impaired and/or would prefer to receive audible guidanceto the pick-up location 108, the server 202 and/or the mobile computingdevice 200 would provide audible guidance 110 for the path 106 from theuser's current position to pick-up location 108, where the autonomousvehicle 104 is situated.

FIGS. 2B and 9B show in step 910 b, when the mobile device locationdetection system 610 outputs a position signal similar to and/or withina threshold distance from the pick-up location 108, the autonomousvehicle 104 audibly outputs the first audible authentication signal 204a.

At step 912 b, after the autonomous vehicle 104 outputs the firstaudible authentication code 204 a, the user 102 is prompted to respondwith the second audible authentication code 204 b.

Next, at step 914 b, the microphone 518 may then receive the audiblyoutput second audible authentication code 204 b and either process thematching authentication onboard the autonomous vehicle 104 or send theaudible authentication signals 204 a, 204 b from the autonomous vehicletransceiver 512 to the server communication system 712, where the server202 uses the pairing system 706 to process and identify whether theaudible authentication signals 204 a, 204 b match. If the latter, theserver communication system 712 then notifies the autonomous vehicle 104whether the audible authentication signals 204 a, 204 b match.

Step 916 b shows that if the audible authentication codes 204 a, 204 bdo not match, the autonomous vehicle 104 a is not the autonomous vehicle104 that the user 102 is searching for. Thus, the user 102 does notboard the autonomous vehicle 104 a and is prompted to search nearby forthe intended autonomous vehicle 104. For example, the autonomous vehicle104 may issue a signal, such as a honk, to indicate that the autonomousvehicle 104 is incorrect. In some embodiments, the intended autonomousvehicle 104 may issue a signal, such as a honk, to provide the user 102some notice of where the autonomous vehicle 104 is located. Again,audible guidance 110 may be provided to the user 102. In someembodiments, the mobile computing device 200 of the user 102 couldnotify the user 102 in some way that is consistent with the user'sunderstanding that there is a mismatch with the autonomous vehicle 104.The mismatch may also be verified by the server 202 and sent to themobile computing device 200. In some embodiments, the mobile computingdevice 200 may be a smart braille device, such that the smart brailledevice can communicate to the user 102 the match or mismatch with theautonomous vehicle 104.

Step 918 b shows that if the audible authentication signals 204 a, 204 bmatch, then the user 102 is allowed to board the autonomous vehicle 104.In some embodiments, the mobile computing device 200 of the user 102 maycommunicate securely with the autonomous vehicle 104 either directlyand/or through the server 202. Furthermore, when the user 102 is allowedto board the autonomous vehicle 104, the autonomous vehicle 104 maynotify the user 102. In some embodiments, the autonomous vehicle 104 mayunlock the doors, play a welcome message, welcome the customer with apersonalized message verifying the name of the customer and/or thedestination 402 etc. In some embodiments, the autonomous vehicle 104 mayrecognize that the user 102 has come with additional passengers andwelcomes the additional passengers as well. Step 920 b shows the user102 boarding the autonomous vehicle 104.

It is further contemplated that the audible authentication codes 204 a,204 b need not both be audible. For example, the autonomous vehicle 104may output the first audible authentication code 204 a so that the user102 may authenticate the first audible authentication code 204 a againsta non-audible authentication code (e.g. haptic-based code that matches apattern of the first audible authentication code, or a visibleauthentication code for non-visually impaired users, etc.). The mobilecomputing device 200 would then send the signal to the server 202informing the server 202 of the successful authentication. The server202 may then send instructions to the autonomous vehicle 104 to unlock.

Many other variations of authentication are further within the scope ofthis disclosure. For example, the autonomous vehicle 104 may output thefirst audible authentication signal 204 a, while the mobile computingdevice 200 is configured to and receives the first audibleauthentication signal 204 a. The mobile computing device 200 may thenencode the first audible authentication signal 204 a and transmit thefirst audible authentication signal 204 a to the server 202, where theserver 202 can authenticate and notify both the autonomous vehicle 104and the mobile computing device 200 if there is a successfulauthentication. Similarly, the mobile computing device 200 may insteadoutput the first audible authentication signal 204 a, while theautonomous vehicle 104 is configured to and receives the first audibleauthentication signal 204 a. The autonomous vehicle 104 may then encodeand transmit the first audible authentication signal 204 a to the server202, where the server 202 can authenticate and notify both theautonomous vehicle 104 and the mobile computing device 200 if there is asuccessful authentication. In these instances, only the first audibleauthentication code 204 a is necessary for a successful authentication.

FIG. 10 provides an exemplary timeline of different processes, whilealso providing insight into which component may handle the process.However, the timeline and categorization of processes by each componentis merely provided as an example and it is to be understood that theorder and handling of each process by different components may be variedin a wide variety of different ways.

As discussed above, the user 102 may request a ride in step 1002. Morespecifically, the user 102 may use the support application 606 on his orher mobile computing device 200 to request a ride on the autonomousvehicle 104. The request is sent through the mobile device transceiver608 to the server communication system 712.

In step 1004, the server 202 then accesses the user profile in the datastore 714 to determine user profile information, such as if the user 102is differently abled and/or if the user 102 has requested thatinformation be provided audibly. The request is then handled by thepairing system 706 to pair the user 102 with the autonomous vehicle 104.The pairing system 706 then creates and sends authentication signals toboth autonomous vehicle transceiver 512 and the mobile devicetransceiver 608 of the mobile computing device 200 of the user 102. Ifthe user 102 is visually impaired or has noted in the profile thatinformation should be provided audibly, then the authentication signalswill be the audible authentication signals 204 a, 204 b.

In step 1006 a, the autonomous vehicle 104 receives the first audibleauthentication signal 204 a. In step 1006 b, the user 102 and/or themobile computing device 200 of the user 102 receives the second audibleauthentication signal 204 b to verify when authenticating the autonomousvehicle 104.

Step 1008 shows the autonomous vehicle arriving at the pickup location108.

Step 1010 shows that, in response to the autonomous vehicle 104 arrivingat the pick-up location 108, the server 202 is notified of the arrivalof the autonomous vehicle 104 by either a notification sent from theautonomous vehicle 104 to the server communication system 710 and/or amatching of the location signal from the autonomous vehicle locationdetection system 514 with the pickup location 108. The server 202 thennotifies the user 102 of the arrival of the autonomous vehicle 104 and,in accordance with the user's profile, causes the mobile device audiosystem 612 to output and provide audible guidance 110 to the pickuplocation 108, where the autonomous vehicle 104 is also located.Alternatively or additionally, the autonomous vehicle 104 may directlycommunicate with the mobile computing device 200.

As stated above, the user's 102 location may be determined through themobile device location detection system 610. The mobile device locationdetection system 610 outputs a position signal that identifies where theuser 102 is located. The mobile computing device 200 may be configuredto send via the mobile device transceiver 608 the position signal outputfrom the mobile device location detection system 610 to the server 202.

Thus steps 1012 and 1014 show that as the user 102 arrives at the pickuplocation 108, where the autonomous vehicle 104 is also located, theserver 202 will detect that the position signal of the mobile devicelocation detection system 610 is similar to and/or within a thresholddistance from the pickup location 108 and the position signal of theautonomous vehicle location detection system 514. The server 202 maythen notify the autonomous vehicle 104 that the user 102 is near theautonomous vehicle 104.

Alternatively, the mobile computing device 200 may communicate directlywith the autonomous vehicle 104. For example, a near-field communication(NFC) system or other wireless technology, such as Bluetooth, may beused to provide direct communication between the mobile computing device200 and the autonomous vehicle 104.

In step 1016, when the autonomous vehicle 104 is aware that the user 102is nearby, either by a notification from the server 202, by directcommunication with the mobile computing device 200 of the user 102, orby any other method of communication, the autonomous vehicle computingsystem 502 will cause the autonomous vehicle audio system 516 to outputthe first audible authentication signal 204 a.

In step 1018, the user 102 verifies the first audible authenticationsignal 204 a against the earlier received second audible authenticationsignal 204 b. In step 1020, the user 102 may then use the mobilecomputing device 200 to notify the server 202 and/or the autonomousvehicle 104 directly through the support application 606 and/or bysending a notification or signal through the mobile device transceiver608. Alternatively, the user 102 and/or the mobile computing device 200of the user 102 may instead output the second audible authenticationsignal 204 b so that the autonomous vehicle 104 receives the secondaudible authentication signal 204 b. The autonomous vehicle 104 may theneither verify onboard the first audible authentication signal 204 aagainst the second audible authentication signal 204 b or send the firstand second audible authentication signals 204 a, 204 b to the server202, where the pairing system 706 determines whether the signals 204 a,204 b match. The server communication system 712 then notifies theautonomous vehicle 104 of a successful authentication.

Step 1022 illustrates that in response to the successful authenticationof the first and second audible authentication signals 204 a, 204 b, theautonomous vehicle control system 510 allows entry to the autonomousvehicle 104 and the user 102 accordingly boards the autonomous vehicle104. It is further contemplated that the autonomous vehicle 104 maypreviously have had doors locked to prevent entry thereto; thus, whenthe autonomous vehicle 104 allows entry thereto, the doors may beunlocked.

FIG. 10 is simply one example in a wide variety of different methodsthat fall within the scope of this disclosure. One of ordinary skill inthe art would understand that modifications to ordering, timing,components, and/or other processes would fall within the scope of thisdisclosure.

FIG. 11 illustrates a schematic block diagram of a procedure 1100 forboarding the autonomous vehicle 104. Procedure 1100 begins with step1105, and continues to step 1110 where the autonomous vehicle 104determines that information is to be provided audibly to the user 102when information is provided to the user 102.

Procedure 1100 continues to step 1115 where the user 102 and theautonomous vehicle 104 create a successful authentication using thefirst audible authentication code 204 a and the second audibleauthentication code 204 b. The successful authentication may be createdin a wide variety of methods, as discussed above.

At step 1120, responsive to the successful authentication, and inaccordance with the profile of the user 102, the user 102 is audiblynotified of the successful authentication. It is to be clear that any ofthe above-identified audio systems or communication systems may be usedto audibly notify the user 102, including but not limited to theautonomous vehicle audio system 516, the mobile device audio system 612,and/or the server communication system 712.

The user 102 is then allowed entry into the autonomous vehicle at step1125. The process subsequently adds at step 1130.

Part 2: En Route

After the user 102 has boarded the autonomous vehicle 104, theautonomous vehicle 104 and the user 102 may begin the journey.

FIG. 12 illustrates the en route process 808.

As shown in step 1202, to ensure that the user 102 to the intendeddestination 402, the server 202 may use the route planning system 708 tocreate the predetermined journey route 302. The server 202 communicatesthe predetermined journey route 302 to the autonomous vehicle 104.Alternatively or additionally, the autonomous vehicle 104 may have alocally stored route planning system to create the predetermined journeyroute 302 and the locally generated predetermined journey route 302 maybe transmitted to the server 202 (e.g., the verification system 710).

In step 1204 and 1206, the autonomous vehicle 104 begins moving alongthe predetermined journey route 302. During the journey, the autonomousvehicle location detection system 514 outputs the position signal of theautonomous vehicle 104. The server communication system 712 may receivethe position signal output, store the position signal output in the datastore 714, and use the position signal output in the verification system710. Alternatively or additionally, the mobile device location detectionsystem 610 may also output the position signal output of the mobiledevice 200, which allows the server communication system 712 tosimilarly receive the position signal output, store the position signaloutput in the data store 714, and use the position signal output in theverification system 710.

Steps 1206 and step 1208 show the verification system detectingdivergences 306 from the predetermined journey route 302. Morespecifically, the verification system 710 compares the position signaloutput of the autonomous vehicle 104 and/or the mobile computing device200 against the predetermined journey route 302. The position signaloutput of the autonomous vehicle 104 creates the actual journey route304, which may or may not be substantially similar to the predeterminedjourney route 302.

Step 1210 shows that if the actual journey route 304 is substantiallysimilar to the predetermined journey route 302, then the autonomousvehicle 104 will arrive at the destination 402 as planned.

On the other hand, if the actual journey route 304 is not substantiallysimilar to and/or exceeds a threshold distance away from thepredetermined journey route 302, then there is the divergence 306.

Step 1212 then shows that the user 102 is notified of the divergence306. More specifically, the verification system 710 of the server 202may communicate with the server communication system 712 to send asignal to the mobile device transceiver 608. The mobile computing device200 processes the signal from the mobile device transceiver 608 and maynotify the user 102 through the support application 606 or any othersuitable method. Alternatively or additionally, the autonomous vehicle104 may announce the divergence through the autonomous vehicle audiosystem 516. In either instances, the audio systems 516, 612 output thesupport message 308, which explains that divergence 306 has occurred.

Step 1214 shows that when the divergence 306 is detected, the server 202may further determine and/or authenticate the cause of the divergence306. This determination and/or authentication may be achieved through awide variety of different methods according to this subject disclosure.

One such method is comparing the position signal outputs of theautonomous vehicle 104 and the position signal outputs of the mobilecomputing device 200 of the user 102. When the position signal outputsdiffer significantly, there is a possibility that the autonomous vehicle104 and/or the mobile computing device 200 has been maliciouslyattacked. In other words, the autonomous vehicle and/or the mobilecomputing device 200 may be compromised to cause the location detectionsystems 508, 610 to output inaccurate position signals (i.e. spoofing).

Another method is comparing the actual journey route 304 against journeyroutes or paths of other nearby autonomous vehicles 104 a. If the actualjourney route 304 differs significantly against other journey routes ofother autonomous vehicles 104 a, there is a possibility that theautonomous vehicle 104 has been malicious attacked. In other words, whenthe autonomous vehicle 104 makes a divergence 306 from the predeterminedjourney route 302 that other autonomous vehicles 104 a do not also do orfollow, the autonomous vehicle 104 may be compromised such that thepropulsion system 522, braking system 524, and/or steering system 526 nolonger follow routing instructions from the server 202. Thus, a thirdparty may have control of the autonomous vehicle 104, resulting inactions not requested and/or implemented by the user 102 and/or theserver 202. In some scenarios, the server 202 and/or the autonomousvehicle 104 may communicate with other autonomous vehicles 104 and/orother sensors nearby to determine whether unusual changes. These sensorsmay include, but are not limited to, other trusted autonomous vehicles104, cameras as part of infrastructure, other government andnon-government infrastructure, etc.

While both methods may provide information of the possibility of amalicious attack, there may be situations in which a malicious attackmay be determined, when in reality there is no malicious attack (i.e. afalse positive malicious attack). To remedy this, a combination of theabove identified methods and/or any other different method may be usefulto authenticate the presence of a malicious attack.

While the above methods are specifically identified, other methods arealso contemplated and within this subject disclosure. For example, inareas having an infrastructure designed with transceivers or otherwireless capabilities (i.e. “smart city”), the server 202 and/or theautonomous vehicle 104 may communicate with the city to determinewhether other autonomous vehicles 104 a are making similar pathingdecisions or if there are new traffic developments, such asconstruction, flooding, etc.

Step 1216 shows that if the verification system 710 determines that thedivergence 306 is not caused by a malicious attack, the verificationsystem 710 communicates with the route planning system 708 to create anew journey route, which becomes the predetermined journey route 302. Inother words, if the verification system 710 determines that theautonomous vehicle 104 diverges 306 from the predetermined journey path302 due to an innocent cause (e.g., missed a turn, could not switchlanes, etc.) then the route planning system 708 reroutes the autonomousvehicle 104 to the destination 402, despite the divergence 306. In someembodiments, the autonomous vehicle 104 may detect the divergence 306and request a new journey route from the server 202, which will respondby having the route planning system 708 create a new journey route thatbecomes the new predetermined journey route 302. It is furthercontemplated that the verification system 710 may further determine thecause of the divergence 306 and, in accordance with the profile of theuser, cause the autonomous vehicle audio system 516 or the mobile deviceaudio system 612 to audibly notify the user 102 of the cause of thedivergence 306. For example, the verification system 710 may determinethat construction has caused a road in the predetermined journey routeto be closed; thus the verification system 710 communicates with themobile device audio system 612 via the support application 606 to outputan audible message stating that the divergence 306 (or turn away fromthe street) is due to a road closure caused by construction.

After receiving the new predetermined journey route 302, theverification system 710 again tracks the progress of the autonomousvehicle 104 and/or the mobile computing device 200 along thepredetermined journey route 302 for divergences 306 from thepredetermined journey route 302. If the actual journey route 304 isidentical to the predetermined journey route 302, then the autonomousvehicle 104 and the user 102 will arrive at the destination 402.

Otherwise, as shown in step 1218, if the verification system 710 hasdetermined that there is a malicious attack, the server communicationsystem 712 sends instructions to the autonomous vehicle 104 to stopand/or pull over. The server 202 may also be configured to send anotification or otherwise inform authorities and/or other emergencyservices of the malicious attack, the position of the user 102 and theautonomous vehicle 104.

Alternatively or additionally, the mobile computing device 200 may beconfigured to also compare the progress of the autonomous vehicle 104and/or the mobile computing device 200 along the predetermined journeyroute 302. If the divergence 306 occurs, the mobile computing device 200may be configured to request information regarding the divergence fromthe server 202 and/or the autonomous vehicle 104. The verificationsystem 710 may then respond with the information. In the event of amalicious attack, the user 102 may then contact authorities and/or otherpeople on their own or may request through the support application 606to have the server 202 and/or the autonomous vehicle 104 contactauthorities and/or other resources.

It is further contemplated that the verification system 710 may also beconfigured to cause the autonomous vehicle audio system 516 and/or themobile device audio system 612 to, in accordance with the user'sprofile, audibly output each turn along the predetermined journey path302, irrespective of whether there is any divergence 306.

It is further contemplated that, in some embodiments, the user 102 maywant to change the destination 402 while enroute. Thus, the user 102 mayspeak to the microphone 518 of the autonomous vehicle 104 and/or themobile computing device 200 to ask for the change. The autonomousvehicle 104 may then send the request to the route planning system 708,which will create a new journey path and update the predeterminedjourney path 302 with the new journey path. Again, the user 102 may benotified audibly of the change. In some embodiments, the user 102 mayalso want to add stops and/or quick destinations along the journey tothe destination 402. Again, the user 102 may speak to the microphone 518of the autonomous vehicle 104 and/or the mobile computing device 200 toask for the addition. The autonomous vehicle 104 may then send therequest to the route planning system 708, which will create a newjourney path with the additional stops and update the predeterminedjourney path 302 with the new journey path.

FIG. 13 is a schematic block diagram of a procedure 1300 for boardingthe autonomous vehicle 104. Procedure 1300 begins with step 1305, andcontinues to step 1310 where it is determined that information is to beprovided audibly to the user 102 when information is provided to theuser 102.

Procedure 1300 continues to step 1315 where a location of the passenger102 is compared against the predetermined journey path 302. Thecomparison may detect the divergence 306 from the predetermined journeypath based upon the position signal output by the autonomous vehicle 104and/or the mobile computing device 200.

At step 1320, responsive to detecting the divergence 306 from thepredetermined journey path 302, the divergence 306 is authenticated. Asstated above, the divergence 306 may be authenticated in a wide varietyof different methods to determine whether there is actually a divergence306 from the predetermined journey path 302 and the cause of thedivergence 306. For example, the divergence 306 may be authenticated bycomparing a location of the autonomous vehicle against a location of asecond autonomous vehicle that is trusted. In some embodiments, thesecond autonomous vehicle acts just as a sensor, in that the secondautonomous vehicle is able to determine the location of the autonomousvehicle 104. In some embodiments, there may be multiple additionalautonomous vehicles, all of which may be able to determine the behaviorand location of the autonomous vehicle 104.

Procedure 1300 continues to step 1325, where the cause of the divergence306 from the predetermined journey path 302 is determined to be from amalicious attack. As stated above, there are a wide variety of differentmethods to determine whether the divergence 306 is caused by a maliciousattack.

At step 1330, in accordance with the profile of the user 102, theautonomous vehicle audio system 516 and/or the mobile device audiosystem 612 outputs the support message 308 to notify the passenger ofthe divergence and the cause of the divergence.

Then at step 1335, responsive to the malicious attack, a distress signalis sent to inform authorities of the malicious attack and the divergence306 from the predetermined journey path 302. Again, the distress signalmay be sent from the mobile computing device 200 of the user, the server202, and/or the autonomous vehicle 104. The procedure 1300 subsequentlyends at step 1340.

Part 3: Disembarking

Assuming the user 102 and the autonomous vehicle 104 arrive at theintended destination 402, there may be a disembarking process 810.

FIG. 14 demonstrates one such disembarking process 810.

As discussed above, the autonomous vehicle 104, the server 202 and/orthe mobile computing device 200 may be tracking the location of theautonomous vehicle 104 and/or the mobile computing device 200. Thus, asshown in step 1402, upon arrival near the destination 402, theautonomous vehicle 104 will find a safe location to stop the autonomousvehicle 104 and come to a stop. The location may be considered safebased on a wide variety of different factors, such as vehicular traffic,foot traffic, speed of traffic, weather conditions, etc. When theposition signal of the autonomous vehicle 104 and/or the position signalof the mobile computing device 200 comes to a stop, the server 202 willcompare the stop location against the location of the destination 402.

As step 1404, if the location is substantially similar and/or within athreshold distance, the server communication system sends a notificationto the user 102. The notification may be audible and may be communicatedthrough the autonomous vehicle audio system 516 and/or the mobile deviceaudio system 612. The notification may further include details of thelocation of the autonomous vehicle 104 in relation to the destination402, which may also allow the user 102 to verify that the destination402 is the intended destination. For example, the notification may state“We have arrived at a safe location. We are parked 100 feet in front ofthe post office.”

Step 1406 shows that the autonomous vehicle 104 then scans an areaaround the vehicle and an intended direction for the user 102 todisembark towards. The autonomous vehicle 104 may scan the surroundingarea with the autonomous vehicle sensor system 520. For example, anoptical camera and/or lidar may be configured to observe obstacles andmovement along a sidewalk nearby.

Step 1408 shows the autonomous vehicle 104 determining whether thesurrounding area is safe. For example, the autonomous vehicle 104 maydetermine the surrounding area is not safe because there is thepedestrian 404 is running in the direction of the autonomous vehicle104, which would create a possibility of the pedestrian hitting the doorof the autonomous vehicle 104 when the user 102 opens the door. If theautonomous vehicle 104 determines that the surrounding area is not safe,it continues to scan the surrounding area until there is a safeopportunity for the user 102 to disembark from the autonomous vehicle104. Many factors may be used to determine whether the surrounding areais safe and/or if there is a safe opportunity for the user 102 todisembark. These factors may include, but are not limited to, thepresence and/or absence of pedestrians, animals, high curbs,skateboarders, flooding and/or other obstacles, the speed at whichobjects are moving, the speed at which the user 102 may be able todisembark (i.e. the amount of time necessary for the user 102 to safelydisembark), etc. In some embodiments, the autonomous vehicle 102 maycreate a prediction of where the objects and/or obstacles will be basedon their current positions and velocities and accordingly furtherdetermine the safe opportunity, such that the safe opportunity takesinto account that no other objects will be entering that area orlocation when the user 102 may disembark.

Step 1410 shows that when the autonomous vehicle 104 has detected a safeopportunity to disembark, the autonomous vehicle 104 notifies the userthat there is a safe opportunity to disembark. Again, this may beaccomplished through the autonomous vehicle audio system 516 and/or themobile device audio system 612.

Step 1412 then shows that the autonomous vehicle 104 may guide the user102 to exit the vehicle during the safe opportunity to disembark. Morespecifically, the autonomous vehicle 104 determines directionalinformation to audibly guide disembarking. This directional informationmay be determined from a wide variety of factors, such as the locationof the autonomous vehicle 104, the orientation of the autonomous vehicle104, a direction of the destination 402 in relation to the autonomousvehicle 104, etc. Again, this information may be audibly provided to theuser 102 through the autonomous vehicle audio system 516 and/or themobile device audio system 612. Furthermore, the autonomous vehicleaudio system 516 may be configured to provide spatial audio. In otherwords, the autonomous vehicle audio system 516 may be configured tooutput audio from specified directions, such that the user 102 will beable to easily identify the specified direction to disembark from theautonomous vehicle 104. While this may be helpful to users 102 withvisual impairments, this spatial audio guidance may also be helpful forall users 102 in general. For example, in camp-style seatingarrangements (e.g. seats all facing inwards, such that passengers mayeasily talk to each other, like around a camp fire), the users 102 mayeasily forget and/or lose track of which direction is “left” and/or“right.” Thus, by spatially providing audible guidance through theautonomous vehicle audio system 516, all users 102 in general wouldeasily understand the correct direction to disembark the autonomousvehicle 104.

In some embodiments, step 1412 further shows that the autonomous vehicle104 may instruct or direct the user 102 to perform a procedure prior toexiting the autonomous vehicle. For example, the autonomous vehicle 104may require that the user 102 use the arm distal from the autonomousvehicle 104 door to open the door (i.e. performing a Dutch reach).

Optional step 1414 shows that the autonomous vehicle 104 observes anddetermines whether the user 102 has conducted the requested procedure.Continuing the earlier example, the autonomous vehicle 104 may observethrough an in-cabin camera of the autonomous vehicle sensor system 520to determine whether the user has performed a Dutch reach to open thedoor.

If the autonomous vehicle 104 does not observe and/or determines thatthe user 102 has not conducted the requested procedure, the autonomousvehicle 104 may again instruct or direct the user 102 to perform theprocedure. For example, when a sighted user 102 has not performed aDutch reach, the autonomous vehicle 104 may remain locked and notify theuser 102 again to perform a Dutch reach.

At step 1416, if the autonomous vehicle 104 observes and determines thatthe user 102 has conducted the requested procedure, the autonomousvehicle 104 may then allow the user 102 to disembark from the autonomousvehicle 104 during the safe opportunity. If the safe opportunity is nolonger present, the autonomous vehicle 104 may notify the user 102 thatthe safe opportunity has ended.

At step 1418, as the user 102 disembarks from the autonomous vehicle104, the autonomous vehicle 104 may, in accordance with the profile ofthe user 102, output the audible message 406 exterior to the autonomousvehicle 104, the audible message 406 indicating that the user 102 isdisembarking from the autonomous vehicle 104. This audible message 406may be helpful in alerting nearby pedestrians 404 that the disembarkingof the user 102 may cause an obstruction in their path.

After the user 102 has disembarked from the autonomous vehicle 104, theserver 202 may cause the mobile device audio system 612 to outputaudible guidance from user's location to the destination 402.

FIG. 15 is a schematic block diagram of a procedure 1500 for boardingthe autonomous vehicle 104. Procedure 1500 begins with step 1505, andcontinues to step 1510 where it is determined that information is to beprovided audibly to the user 102 when information is provided to theuser 102.

Procedure 1500 continues to step 1515 where the autonomous vehicle 104searches for a safe location to stop the autonomous vehicle 104. Thesafe location may be directly in front of the destination 402.

At step 1520, in accordance with the profile of the passenger 102, theautonomous vehicle 104 may cause the autonomous vehicle audio system 516to audibly output that the autonomous vehicle has found the safelocation to stop.

Procedure 1500 continues to step 1525, the autonomous vehicle 104 thensearches for and attempts to detect a safe opportunity for the passengerto disembark or exit the autonomous vehicle 104. As stated above, thesafe opportunity may be determined based on a wide variety of differentfactors.

At step 1530, responsive to detecting the safe opportunity, and inaccordance with the profile of the passenger 102, the autonomous vehicleaudio system 516 audibly notifies and guides the passenger 102 to exitthe autonomous vehicle 104 safely during the safe opportunity. As statedabove, the autonomous vehicle audio system 516 may spatially guide thepassenger 102 through directional audio.

Then at step 1535, in accordance with the profile of the passenger 102,the audible message 406 is output, wherein content of the audiblemessage 406 is based on the passenger 102 exiting the autonomous vehicle104 being disabled. The procedure subsequently ends at step 1540.

Various functions described herein can be implemented in hardware,software, or any combination thereof. If implemented in software, thefunctions can be stored on or transmitted over as one or moreinstructions or code on a computer-readable medium. Computer-readablemedia includes computer-readable storage media. A computer-readablestorage media can be any available storage media that can be accessed bya computer. By way of example, and not limitation, suchcomputer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that can be used to store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and blu-ray disc (BD), where disks usually reproducedata magnetically and discs usually reproduce data optically withlasers. Further, a propagated signal is not included within the scope ofcomputer-readable storage media. Computer-readable media also includescommunication media including any medium that facilitates transfer of acomputer program from one place to another. A connection, for instance,can be a communication medium. For example, if the software istransmitted from a website, server, or other remote source using acoaxial cable, fiber optic cable, twisted pair, digital subscriber line(DSL), or wireless technologies such as infrared, radio, and microwave,then the coaxial cable, fiber optic cable, twisted pair, DSL, orwireless technologies such as infrared, radio, and microwave areincluded in the definition of communication medium. Combinations of theabove should also be included within the scope of computer-readablemedia.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Application-specific Integrated Circuits (ASICs),Application-specific Standard Products (ASSPs), System-on-a-chip systems(SOCs), Complex Programmable Logic Devices (CPLDs), etc Similarly,functionality described herein can be performed on different components.For example, and without limitation, determinations by the verificationsystem 710 may be performed onboard the autonomous vehicle 104 and viceversa.

What has been described above includes examples of one or moreembodiments. It is, of course, not possible to describe everyconceivable modification and alteration of the above devices ormethodologies for purposes of describing the aforementioned aspects, butone of ordinary skill in the art can recognize that many furthermodifications and permutations of various aspects are possible.Accordingly, the described aspects are intended to embrace all suchalterations, modifications, and variations that fall within the spiritand scope of the appended claims. Furthermore, to the extent that theterm “includes” is used in either the details description or the claims,such term is intended to be inclusive in a manner similar to the term“comprising” as “comprising” is interpreted when employed as atransitional word in a claim.

What is claimed is:
 1. A mobile computing device comprising: a speaker;a position signal sensor configured to output a position signal of themobile computing device; a processor in communication with the speaker;and memory that stores computer-executable instructions that, whenexecuted by the processor, cause the processor to: determine, based upona profile of a user, that support is to be provided audibly to the userwhen the support is provided to the user; detect divergence from apredetermined path based upon the position signal output by the positionsignal sensor; responsive to detecting divergence from the predeterminedpath, notify a server of the divergence; receive a transmission from theserver, the transmission having an authentication of the divergence; andresponsive to receiving the transmission and in accordance with theprofile of the user, cause the speaker to output an audible supportmessage, wherein content of the audible support message is based on thedetected divergence.
 2. The mobile computing device of claim 1, whereinthe computer-executable instructions, when executed by the processor,further cause the processor to: determine a cause of the divergence fromthe predetermined path to be from a malicious attack; and responsive tothe malicious attack, send a distress signal to inform authorities ofthe malicious attack and the divergence from the predetermined path. 3.The mobile computing device of claim 1, wherein the computer-executableinstructions, when executed by the processor, further cause theprocessor to: communicate with a second autonomous vehicle to compare alocation of a first autonomous vehicle and the position signal.
 4. Themobile computing device of claim 1, wherein the divergence from thepredetermined path is authenticated against paths of nearby vehicles. 5.The mobile computing device of claim 1, wherein the mobile computingdevice is in communication with a server configured to: observe theposition signal of the mobile computing device and a position of anautonomous vehicle; determine the predetermined path; detect thedivergence of the position signal of the mobile computing device or alocation of the autonomous vehicle from the predetermined path; andnotify an observer of the divergence from the predetermined path.
 6. Asystem, comprising a processor; and memory that storescomputer-executable instructions that, when executed by the processor,cause the processor to: determine, based upon a profile of a passenger,that support is to be provided audibly to the passenger when the supportis provided to the passenger; detect divergence from a predeterminedpath based upon a position signal output by a position sensor system ofan autonomous vehicle and a position signal output by a position sensorsystem of a mobile computing device of the passenger; and responsive todetecting divergence from the predetermined path, and in accordance withthe profile of the passenger, transmit a signal to the mobile computingdevice and causing the mobile computing device to output an audiblesupport message, wherein content of the audible support message is basedon the detected divergence.
 7. The system of claim 6, wherein thecomputer-executable instructions, when executed by the processor,further cause the processor to: authenticate the detected divergence ofthe autonomous vehicle from the predetermined path by determining aposition of the mobile computing device.
 8. The system of claim 6,wherein the computer-executable instructions, when executed by theprocessor, further cause the processor to: send a distress signal toinform authorities of the divergence from the predetermined path.
 9. Thesystem of claim 6, wherein the computer-executable instructions, whenexecuted by the processor, further cause the processor to: determine acause of the divergence from the predetermined path to be from amalicious attack.
 10. The system of claim 6, wherein thecomputer-executable instructions, when executed by the processor,further cause the processor to: compare a location of the autonomousvehicle and a location of the mobile computing device of the passengerto determine a false positive malicious attack.
 11. The system of claim6, wherein the computer-executable instructions, when executed by theprocessor, further cause the processor to: stop the autonomous vehiclein response to a malicious attack.
 12. The system of claim 6, whereinthe computer-executable instructions, when executed by the processor,further cause the processor to: authenticate divergence from thepredetermined path against nearby vehicles.
 13. The system of claim 6,wherein the computer-executable instructions, when executed by theprocessor, further cause the processor to: request a new predeterminedpath.
 14. A method comprising: determining, based upon a profile of apassenger of an autonomous vehicle, that support is to be providedaudibly to the passenger when the support is provided to the passenger;detecting divergence from a predetermined path based upon a position ofthe autonomous vehicle; responsive to detecting divergence from thepredetermined path, determining a cause of the divergence from thepredetermined path to be from a malicious attack; and sending a distresssignal to inform authorities of the divergence from the predeterminedpath and the malicious attack.
 15. The method of claim 14, furthercomprising: authenticating the divergence from the predetermined pathagainst a position of a mobile device of the passenger.
 16. The methodof claim 14, further comprising: responsive to the malicious attack, andin accordance with the profile of the passenger, causing a speaker tooutput an audible message, wherein content of the audible message isbased on the detected divergence and the malicious attack.
 17. Themethod of claim 14, further comprising: comparing a location of theautonomous vehicle and a location of a mobile device of the passenger todetermine a false positive malicious attack.
 18. The method of claim 14,further comprising: responsive to the malicious attack, stopping theautonomous vehicle.
 19. The method of claim 14, further comprising:authenticating divergence from the predetermined path against nearbyvehicles.
 20. The method of claim 14, further comprising: requesting anew predetermined path.